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TITLE OF THE INVENTION 

IMAGE PROCESSING 'UNIT AND IMAGE PROCESSING METHOD WITH 
SIMPLIFIED EXPOSURE, GAIN AND WHITE BALANCE CONTROL 

5 

BACKGROUND OF THE INVENTION 
Field of the Invention 

The present invention relates to an image processing unit 
and image processing method of a digital still video camera 
10 that carries out exposure control, gain control and white 
balance control using coded data formed by coding image signals 
output from an image pickup apparatus. 



Description of Related Art 

15 Japanese patent application laid-open No. 4-298170/1992 

discloses a technique about a digital still video camera with 
a function of coding, through orthogonal transformation, 
image signals acquired by an image pickup device such as a 
CCD (charge-coupled device) . This technique utilizes DC 

2 0 (direct current) component data obtained by the orthogonal 
transformation in order to control exposure. 

Fig. 16 is a block diagram showing a configuration of 
a digital still video camera disclosed in the above-mentioned 
Japanese patent application laid-open No. 4-298170. In Fig. 

25 16, the reference numeral 101 designates a diaphragm for 
spatially controlling the amount of light entering the camera; 
102 designates a shutter for temporally controlling the amount 
of the incident light; 103 designates a CCD for converting 
the received incident light to electrical signals; 104 

30 designates a Y/C separator for generating from the image 



2 

signals output from the CCD 103 a luminance component and color 
components to be output separately; 105 designates a CCD driver 
for supplying the CCD 103 with drive signals for driving it; 
106 designates an A/D converter for converting the analog 
5 luminance signal and the color signals output from the Y/C 
separator 104 into digital image data; 107 designates a 
high-speed buffer memory for temporarily storing the image 
data output from the A/D converter 106; 108 designates a data 
compressor/decompressor that reads from the high-speed buff er 

10 memory 107 the image data, divides the image data of an amount 
of a single screen into a plurality of blocks, and carries 
out data compression by coding through quantization of 
frequency information obtained by orthogonal transformation 
of each block, or data expansion by the reverse transformation; 

15 109 designates a recording memory for recording the data 
compressed image data through the data 

compressor/decompressor 108; 110 designates a D/A converter 
for converting the image data read from the high-speed buffer 
memory 107 into an analog signal; and 111 designates a system 

20 controller that supplies the foregoing blocks with timing 
signals for controlling their operation, and outputs 
controlling signals for controlling the diaphragm 101 and the 
shutter 102 by reading as photometric data the DC component 
data of each block after the orthogonal transformation in the 

25 data compression process in the data compressor/decompressor 
108 . 

Next, the operation of the conventional digital still 
video camera will be described. 

In this conventional digital still video camera, the 
30 system controller 111 reads block by block the DC component 



data of each block after the orthogonal transformation in the 
data compression process by the data compressor/decompressor 
108, and carries out the exposure control by controlling the 
driving of the diaphragm 101 and the shutter 102 by using the 
read data as the photometric data, thereby achieving the right 
exposure to a subject of the camera. 

The conventional image processing unit with the 
foregoing structure employs the orthogonal transformation as 
its coding scheme. Thus, calculating and coding the 
photometric data on the real time basis requires a real time 
processing of a series of coding sequences. This, however, 
has a problem of complicating the control and increasing the 
circuit scale because the orthogonal transformation coding 
is one of the variable length coding methods. 

Furthermore, to use the photometric data to implement 
gain correction of the digital data of the picked-up image, 
it is necessary to hold besides the coded data the DC components, 
or to regenerate the DC components by decoding the coded data, 
which presents a problem of increasing the processing 
procedure . 

In addition, since the gain correction of the digital 
data obtained by coding the picked-up image cannot be provided 
directly to the coded data, the coded data must be decoded 
once, and then subjected to the sequential processing- This 
presents another problem of hindering the high speed 
processing. 

Moreover, since the coding based on the orthogonal 
transformation is one of the variable length coding methods, 
the control such as calculating addresses for reading the coded 
data during the decoding becomes complicated, and this 
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presents a problem of reducing the processing speed and 
increasing the circuit scale. 

Furthermore, coding into a single color signal component 
the image output from the image pickup device , which includes 
5 a plurality of color filters, poses a problem of degrading 
the accuracy of the photometric data. This is because the 
color modulation components are each processed as a resolution 
component, which increases the quantization error due to the 
coding . 

10 In addition, it is necessary for the white balance 

processing, which corrects sensitivity errors in individual 
colors of the image output from the image pickup device 
including a plurality of color filters, to have coding circuits 
of the number of the color components, and this presents a 

15 problem of increasing the circuit scale. 

Moreover, to select a block used for determining the 
photometric data in desired conditions, it is necessary for 
the processing of deciding the conditions to utilize data in 
the process of coding rather than the coded data themselves . 

20 This presents another problem of increasing the processing 
procedure . 

Finally, to select a block used for determining the 
photometric data in desired conditions, the block to be used 
for generating the photometric data cannot be selected by a 
25 simple processing because the processing, which carries out 
the gain correction of the digital data obtained by coding 
the picked-up image, employs the variable length coding. This 
presents still another problem of complicating the 
processing . 
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SUMMARY OF THE INVENTION 

The present invention is implemented to solve the 
foregoing problems. It is therefore an object of the present 
invention to provide an image processing unit capable of 
5 achieving a real-time processing of a series of coding 

sequences with a simple control and a small circuit scale. 

Another object of the present invention is to provide 
an image processing unit capable of implementing the gain 
correction using the coded data with a simple processing by 
10 generating only from the coded data the photometric data used 
for the gain correction of the digital image picked-up data, 
without holding the photometric data in addition to the coded 
data . 

Still another object of the present invention is to 
15 provide an image processing unit capable of implementing the 

high-speed gain correction which uses the coded data by 

directly applying the gain correction of the digitally coded 

picked-up image data to the coded data. 

Another object of the present invention is to provide 
20 an image processing unit capable of facilitating the control 

such as calculation of the addresses for reading the coded 

data during decoding. 

Still another object of the present invention is to 

provide an image processing unit capable of improving the 
25 accuracy of the photometric data by reducing the quantization 

errors involved in coding the output image of the image pickup 

device including a plurality of color filters because the color 

modulation components are processed as a resolution 

component . 

30 Another object of the present invention is to provide 



6 



an image processing unit capable of achieving, with a small 
circuit scale , the white balance processing for correcting 
sensitivity errors in ' individual colors of the image output 
from the image pickup device which includes the plurality of 
5 color filters, without using coding circuits of the number 
of the color components. 

Still another object of the present invention is to 
provide an image processing unit capable of reducing the 
processing procedure to select a block used for determining 

10 the photometric data in desired conditions, by using the coded 
data themselves to decide the conditions. 

Another object of the present invention is to provide 
an image processing unit capable of determining, when 
selecting a block to be used for generating the photometric 

15 data, the block used for generating the photometric data with 
simplifying the processing of carrying out the gain correction 
of the digitally coded picked~up image. 

According to a first object of the present invention, 
there is provided an image processing unit comprising: an 

20 A/D converter for carrying out A/D conversion of image signals 
output from an image pickup apparatus that picks up an image 
and converts it into electrical signals, and for outputting 
A/D converted image signals as image data; a fixed length 
coding circuit for dividing the image data into unit blocks, 

25 each consisting of a predetermined number of pixels, and for 
coding the pixels in each unit block after obtaining an average 
level of the pixels in the unit block; and an exposure 
controller for calculating a luminance level of a whole set 
of pixel data in the unit blocks by integrating the average 

30 levels of the unit blocks, and for controlling exposure of 



the image pickup apparatus such that the luminance level of 
the image data matches a predetermined level. 

Here, the image processing unit may further comprise a 
coded image memory for storing fixed length coded data output 
5 from the fixed length coding circuit, the fixed length coded 
data including the average levels of the unit blocks. 

The image processing unit may further comprise a fixed 
length decoding circuit for reading from the coded image memory 
the fixed length coded data, for calculating a luminance level 

10 of the image data from the average levels, and for carrying 
out fixed length decoding of the fixed length coded data with 
performing gain correction for adjusting the luminance level 
of the image data to a predetermined level. 

The image processing unit may further comprising: a 

15 fixed length decoding circuit for reading from the coded image 
memory the fixed length coded data, and for carrying out fixed 
length decoding of the fixed length coded data; and a signal 
processor for carrying out, using the average levels, gain 
correction of image data output from the fixed length decoding 

20 circuit. 

According to a second aspect of the present invention, 
there is provided an image processing unit comprising: an 
A/D converter for carrying out A/D conversion of image signals 
output from an image pickup device that picks up an image and 

25 converts it into electrical signals including a plurality of 
color components, and for outputting A/D converted image 
signals as image data; a pixel rearrangement circuit for 
sorting the image data output from the A/D converter such that 
each color component is arranged in a unit block, each of which 

30 consists of a predetermined number of pixels; and 
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a fixed length coding circuit for coding the pixels in each 
unit block after obtaining an average level of the pixels in 
the unit block. 

Here, the image processing unit may further comprise an 
5 exposure controller for calculating a luminance level of a 
whole set of pixel data in the unit blocks by integrating the 
average levels of the unit blocks, and for controlling exposure 
of the image pickup apparatus such that the luminance level 
of the image data matches a predetermined level. 

10 The image processing unit may further comprise signal 

level correction means for correcting, using the average 
levels of the unit blocks, relative signal levels of the 
respective color components. 

The image processing unit may further comprise a coded 

15 image memory for storing fixed length coded data including 
the average level of each unit block, the fixed length coded 
data being output from the fixed length coding circuit. 

The signal level correction means may read from the coded 
image memory the average levels, calculate from the average 

20 levels luminance levels of the pixel data in the unit blocks 
of the respective color components, and correct signal levels 
of the color components such that the luminance levels of the 
color components match with each other. 

The image processing unit may further comprise a fixed 

25 length decoding circuit for reading from the coded image memory 
the fixed length coded data, and for carrying out fixed length 
decoding of the fixed length coded data with performing gain 
correction for adjusting, for each color component, the 
luminance level of the image data to a predetermined level 

30 by using the average levels. 



The image processing unit may further comprise: a fixed 
length decoding circuit for reading from the coded image memory 
the fixed length coded data, and for carrying out fixed length 
decoding of the fixed length coded data; and a signal processor 
for carrying out, for each color component, signal level 
correction of image data output from the fixed length decoding 
circuit by using the average levels. 

The image processing unit may further comprise selecting 
means for selecting a number of the unit blocks. 

The image processing unit may further comprise selecting 
means for selecting a location of the unit blocks. 

The image processing unit may further comprise selecting 
means for selecting the unit blocks in accordance with their 
average levels . 

According to a third aspect of the present invention, 
there is provided an image processing method of reproducing 
a picked~up still image, the method comprising the steps of: 
carrying out A/D conversion of image signals output from an 
image pickup apparatus, and for outputting A/D converted image 
signals as image data; dividing the image data into unit blocks, 
each consisting of a predetermined number of pixels, and coding 
the pixels in each unit block after obtaining an average level 
of the pixels in the unit block; and calculating a luminance 
level of a whole set of pixel data in the unit blocks by 
integrating the average levels of the unit blocks, and 
controlling exposure of the image pickup apparatus such that 
the luminance level of the image data matches a predetermined 
level . 



BRIEF DESCRIPTION OF THE DRAWINGS 
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Fig. 1 is a block diagram showing an embodiment 1 of an 
image processing unit in accordance with the present 
invention; 

Fig. 2 is a diagram showing an arrangement of color 
filters used by the image pickup device of the embodiment 1; 

Fig. 3 is a diagram illustrating quantization levels for 
scaling the intensity of image signals of each pixel with a 
fixed length coding circuit; 

Figs- 4 and 5 are a flowchart showing a coding procedure 
of the embodiment 1 ; 

Fig. 6 is a block diagram showing an embodiment 2 of the 
image processing unit in accordance with the present 
invention; 

Fig. 7 is a block diagram showing an embodiment 3 of the 
image processing unit in accordance with the present 
invention; 

Figs. 8 and 9 are a flowchart showing a procedure of 
calculation for the gain correction carried out during 
decoding by the embodiment 3 of the image processing unit; 

Fig. 10 is a block diagram showing an embodiment 4 of 
the image processing unit in accordance with the present 
invention; 

Fig. 11 is a block diagram showing an embodiment 5 of 
the image processing unit in accordance with the present 
invention; 

Fig. 12 is a schematic diagram illustrating a principle 
of a white balance processing for correcting levels of 
individual color signals; 

Fig. 13 is a block diagram showing an embodiment 6 of 
the image processing unit in accordance with the present 



invention; 

Fig. 14 is a block diagram showing an embodiment 7 of 
the image processing unit in accordance with the present 
invention; 

Fig. 15 is a block diagram showing an embodiment 8 of 
the image processing unit in accordance with the present 
invention; and 

Fig. 16 is a block diagram showing a digital still video 
camera disclosed in Japanese patent application laid-open No. 
4-298170/1992. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

The invention will now be described with reference to 
the accompanying drawings . 
EMBODIMENT 1 

Fig. 1 is a block diagram showing an embodiment 1 of an 
image processing unit in accordance with the present invention. 
In Fig. l,the reference numeral 1 designates an image pickup 
apparatus including a one-chip image pickup device such as 
a CCD that includes color filters with a plurality of colors, 
and outputs pixel signals in a dot serial manner; 2 designates 
an analog signal processing circuit comprising a filtering 
circuit, and an amplifier for amplifying analog image signals 
output from the image pickup apparatus 1 to carry out exposure 
control; 3 designates an A/D converter for converting analog 
signals output from the analog signal processing circuit 2 
into digital signals; 5 designates a fixed length coding 
circuit for performing a fixed length coding of the digital 
image data on a block by block basis of the color signals output 
from the A/D converter 3, the coding being one of FBTC (fixed 



block truncation coding) schemes; 6 designates a coded image 
memory for storing coded data output from the fixed length 
coding circuit 5; 17 designates a driver circuit for 
controlling the exposure time of the image pickup device of 
the image pickup apparatus 1 for carrying out exposure control; 
and 18 designates a system controller for controlling the 
analog signal processing circuit 2 and the driver circuit 17 
in accordance with the coded data output from the fixed length 
coding circuit 5 . 

The reference numeral 7 designates a fixed length 
decoding circuit for carrying out fixed length decoding by 
reading the coded data from the coded image memory 6; 9 
designates a signal processing circuit for performing, on the 
image data decoded by the fixed length decoding circuit 7, 
pixel interpolation, gray level correction, gamma correction 
and the like; 10 designates a frame memory for storing signals 
corrected by the signal processing circuit 9; 11 designates 
a variable length coding circuit for coding, for secondary 
storage, data read from the frame memory 10 using a variable 
length coding scheme like JPEG (Joint Photographic Experts 
Group) schemes; 12 designates a display unit 12 like a CRT 
for displaying data read from the frame memory 10 as an image; 
and 13 designates a secondary storage such as a floppy disk, 
hard disk or flash memory. 

Next, the operation of the present embodiment 1 will be 

described. 

A fundamental processing of a common AE (Auto-Exposure) 
for controlling the exposure of the image pickup device in 
the image pickup apparatus 1 is carried out by setting pickup 
conditions such that the luminance level of the picked-up image 
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matches a predetermined level. The luminance level of the 
image is obtained by integrating the image signals. To achieve 
the integration digitally, it is necessary to sequentially 
accumulate the A/D converted image data over a predetermined 
5 number of pixels. 

In the present embodiment 1, the image pickup apparatus 
1 captures images of a subject with its pickup device, and 
outputs, pixel by pixel in the scanning direction in a 
dot-serial manner, image signals corresponding to incident 

10 light filtered through the color filters. The color image 
signals are amplified by the analog signal processing circuit 
2, and subjected to filtering for removing noise components. 
The image signals are converted into digital signals through 
the A/D converter 3, and are output as digital image data. 

15 The fixed length coding circuit 5 carries out the fixed length 
coding of the digital image data on a block by block basis 
consisting of 4 by 4 pixels, that is, the total of 16 pixels 
as shown in Fig. 2. The unit block includes the three colors 
RGB in a mixed state, and the fixed length coding is carried 

20 out in this state. 

Fig. 3 illustrates quantization levels for the fixed 
length coding circuit 5 to quantize the intensity of image 
signals of each pixel. In Fig. 3, Lmin is a minimum value 
of the image signal intensity in the 16 pixels as shown in 

25 Fig. 2, Lmax is a maximum value of the image signal intensity 
in the same 16 pixels, PI is a value at the bottom one of eight 
equal divisions between the maximum value Lmax and the minimum 
value Lmin, P2 is a value at the top one of the eight divisions, 
Ql is the average value of the pixels with the intensity equal 

30 to or greater than Lmin and equal to or less than PI, and Q8 
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is the average value of pixels with the intensity equal to 
or less than Lmax and greater than P2 . 

The symbol LD is a gray scale width indicator in the unit 
block, which is equal to Q8-Q1, and L1-L7 are subdivisions 
5 of the gray scale width indicator LD, which are obtained by 
dividing the indicator by a factor of 8, and arranged in 
ascending order. The symbol LA is the average level of the 
image data in the unit block, and is equal to (Ql + Q8>/2, 
and 0 ijk represents the quantization level of each pixel. 

10 Figs. 4 and 5 are a flowchart showing a coding procedure 

of the embodiment 1. The coding procedure will be described 
with reference to the flowchart. 

First, the fixed length coding circuit 5 reads the image 
data in a unit block which is converted into the digital form 

15 through the A/D converter 3 (stepSTl). Then, the fixed length 
coding circuit 5 calculates the signal intensity of the image 
data of the 4X4 pixels, and successively obtains the values 
PI, P2, Ql, Q8, LA, LD, L1-L3 and L5-L7 (steps ST2-ST13) using 
the following equations. 



PI 




(Lmax + 7Lmin) / 


P2 




(7 Lmax + Lmin) / 


Ql 




Ave (Xmn S PI) 


Q8 




Ave (Xmn > P2 ) 


LA 




(Ql + Q8)/2 


LD 




Q8 - Ql 


LI 




LA - 3LD/8 


L2 




LA - LD/4 


L3 




LA - LD/8 


L5 




LA + LD/8 


L6 




LA + LD/4 
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L7 = LA + 3LD/8 

Here, Ql equation expresses obtaining the average value 
of the pixels with the intensity equal to or greater than Lmin 
and equal to or less than PI, and Q8 equation describes 
5 obtaining the average value of pixels with the intensity equal 
to or less than Lmax and greater than P2 . 

After successively obtaining the values PI, P2, Ql, Q8, 
LA, LD, L1-L3 and L5-L7 in this way, the fixed length coding 
circuit 5 places n=l and m=l (steps ST14 and ST15) , and decides 
10 as to whether the signal intensity (referred to as a ^pixel 
value" from now on) Xmn of the pixel address (m,n) (that is, 
Xll in the present case) is equal to or less than LI (step 
ST16) . 

If the pixel value Xll is equal to or less than LI, the 
15 fixed length coding circuit 5 places the quantization level 
0 ijk of the pixel at a binary value 000 (step ST17) . Then, 
incrementing m by one (step ST31) , the fixed length coding 
circuit 5 makes a decision as to whether m is equal to or less 
than four (step ST32) . If m is equal to or less than four, 
20 the fixed length coding circuit 5 compares the pixel value 
of the pixel with Ll, again (step ST16) . 

If m is greater than four, the fixed length coding circuit 
5 increments n by one (step ST33) , and makes a decision whether 
the incremented n is equal to or less than four (step ST34) . 
25 If n is equal to or less than four, the fixed length coding 
circuit 5 compares the pixel value of the pixel with Ll again 
(step ST16) . 

If the pixel value Xll is greater than Ll, the fixed length 
coding circuit 5 decides whether it is equal to or less than 
30 L2 (step ST18) . If the pixel value Xmn is equal to or less 
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than L2, the fixed length coding circuit 5 sets the 
quantization level 0 ijk of the pixel at a binary value 001 
(step ST19) . Then, incrementing m by one (step ST31) , the 
fixed length coding circuit 5 makes a decision as to whether 
5 m is equal to or less than four (step ST32) . If m is equal 
to or less than four, the fixed length coding circuit 5 compares 
the pixel value of the pixel with LI, again (step ST16) - If 
m is greater than four, the fixed length coding circuit 5 
increments n by one (step ST33) , and makes a decision whether 
10 the incremented n is equal to or less than four (step ST34) ♦ 
If n is equal to or less than four, the fixed length coding 
circuit 5 compares the pixel value of the pixel with LI again 
(step ST16) . 

Thus, the fixed length coding circuit 5 decides the 

15 division L1-L2, L2-L3, L3-LA, LA-L5, L5-L6 or L6-L7 , to which 
the pixel value belongs (steps ST16, ST18, ST20, ST22, ST24, 
ST26 and ST28) , and assigns to the pixel one of the quantization 
levels 0 ijk - 000, 001, 010, 011, 100, 101, 110 and 111 (steps 
ST17, ST19, ST21, ST23, ST25, ST27 and ST29) . 

20 The coding is completed by thus assigning the 

quantization levels to all the pixels in the unit block. The 
coded data of the unit block consist of LA, LD and 0 ijk of 
respective pixels . 

The foregoing process is iterated by the number of the 

25 unit blocks in the entire image. 

The luminance level of the entire image can be obtained 
by a small amount of calculations by integrating the average 
levels LA of the fixed length coded data of the individual 
unit blocks obtained through the fixed length coding by the 

30 fixed length coding circuit 5. This is because the average 



17 



levels LA each amount to the information obtained by averaging 
the image signal intensity of all the 16 pixels in the unit 
block, which means that the accumulation processing of the 
16 pixels has been completed, and hence the integrating 
5 processing can be reduce by a factor of 16. The system 
controller 18 obtains the luminance level of the image by 
integrating the average levels LA, and controls the driver 
circuit 17 and analog signal processing circuit 2 to achieve 
an intended luminance level . 

10 The fixed length coding circuit 5 stores the fixed length 

coded data in the coded image memory 6 . To reproduce the image, 
the fixed length decoding circuit 7 reads the fixed length 
coded data stored in the coded image memory 6, and carries 
out the fixed length decoding of the data. Subsequently, the 

15 signal processing circuit 9 performs on the decoded data the 
pixel interpolation, the gray level correction, the gamma 
correction and the like, and stores the data into the frame 
memory 10. When displaying the image, the display unit 12 
reads the image signals from the frame memory 10, and displays 

20 the image. To store the data in the secondary storage 13, 
the variable length coding circuit 11 carries out the variable 
length coding of the data, and stores the data in the secondary 
storage 13. 

As described above, the present embodiment 1 can obtain 
2 5 the luminance level of the entire image by a small amount of 
calculations, thus offering an advantage of being able to 
perform real time exposure control with ease. 

EMBODIMENT 2 

30 Fig. 6 is a block diagram showing an embodiment 2 of the 
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image processing unit in accordance with the present invention, 
in which the same reference numerals designate the same 
components as those of the foregoing embodiment 1 as shown 
in Fig. 1, and the description thereof is omitted here. 
5 Next, the operation of the present embodiment 2 will be 

described. 

In the present embodiment 2, the fixed length coded data 
output from the fixed length coding circuit 5 is temporarily 
stored in the coded image memory 6. The system controller 

10 18 reads from the coded image memory 6 the average levels LA, 
and controls the exposure in accordance with the average levels 
LA. The exposure control scheme of the system controller 18 
is the same as that of the foregoing embodiment 1. 

As described above, the present embodiment 2 stores in 

15 the coded image memory 6 the coded data of the entire image. 
This offers an advantage of being able to achieve the exposure 
control which can set the integration conditions flexibly 
using software or the like. 

20 EMBODIMENT 3 

Fig. 7 is a block diagram showing a configuration of an 
embodiment 3 of the image processing unit in accordance with 
the present invention. In Fig. 7, the same reference numerals 
designate the same components as those of the embodiment 1 

25 as shown in Fig. 1, and the description thereof is omitted 
here. In Fig. 7, the reference numeral 7a designates a fixed 
length decoding circuit for carrying out decoding with a gain 
correction function . 

When capturing pictures under fluorescent lamps or 

30 shooting moving subject, the pickup conditions during 



photometry can sometimes vary from those during picking up 
the image. In such a case, the image data must undergo gain 
correction so that the calculated luminance level of the 
captured image matches a predetermined level. The present 
embodiment 3 of the image processing unit performs such gain 
correction. The calculation of the luminance level of the 
image data can be carried out using the average levels LA of 
the fixed length coded data as in the embodiments 1 and 2. 

Next, the operation of the present embodiment 3 will be 
described. 

In the present embodiment 3, the fixed length coded data 
generated by the fixed length coding circuit 5 is temporarily 
stored in the coded image memory 6. 

A gain correction coefficient K is given by K=S/C, where 
S is the predetermined luminance level, and C is the luminance 
level of the picked-up image data. Therefore, the luminance 
level of the reproduced image can be adjusted to the prescribed 
luminance level by multiplying all the image data acquired 
through the image pickup apparatus 1 by the gain correction 
coefficient K. In addition, the amount of calculation for 
the gain correction can be reduced by carrying the calculation 
during decoding. 

Figs. 8 and 9 are a flowchart illustrating the gain 
correction calculation during the decoding. The decoding 
procedure by the fixed length decoding circuit 7a will now 
be described with reference to the flowchart of Figs. 8 and 
9. 

First, the system controller 18 reads from the coded image 
memory 6 the average level LA and the gray scale width indicator 
LD of the current unit block, and supplies them to the fixed 
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length decoding circuit 7a along with the gain correction 
coefficient K. Here, the gray scale width indicator LD is 
that of the unit blocks, which is equal to Q8-Q1. 

From the average level LA, the gray scale width indicator 
LD and the gain correction coefficient K, the fixed length 
decoding circuit 7a obtains in advance decode levels 
LEVEL0-LEVEL7 which are multiplied by the gain correction 
coefficient K (step ST81-ST88) . 

Subsequently, the fixed length decoding circuit 7a 
places a vertical coordinate value n to one (step ST89) , and 
then a horizontal coordinate value m to one (step ST90) . Thus, 
the coordinates (1,1) are designated as an address in the unit 
block by the steps ST89 and ST90. 

Next, the fixed length decoding circuit 7a decides the 
quantization level <£> ij k of the designated address (steps ST91, 
ST93, ST95, ST97, ST99, ST101 and ST103), and assigns, in 
accordance with the decided quantization level <t> ijk, one of 
the gain corrected decode levels LEVELO— LEVEL7 as the signal 
intensity Ymn of the pixel (Yll, if the pixel coordinates are 
(1,1)) (steps ST92, ST94, ST96, ST98, ST100, ST102, ST104 and 
ST105) . 

After obtaining the signal intensity of the pixel (1,1) , 
the fixed length decoding circuit 7a increments the position 
of the pixel by one in the horizontal direction (step ST106) , 
and decodes the signal intensity of the pixel (2,1) through 
the same procedure (step ST91-105) . 

After decoding the signal intensity of the pixels in the 
top row in the unit block (step ST107), the fixed length 
decoding circuit 7a increments the coordinate in the vertical 
direction by one (stepST108), and decodes the signal intensity 
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of the pixels in the next row through the same procedure (step 
ST90-107) . 

By thus decoding all the pixels in the unit blocks (steps 
ST90-ST109) , the decoding is completed. 
5 According to the present embodiment 3, even when the 

signal intensity of the 16 pixels ranges over all the eight 
decode levels LEVELO— LEVEL7 , for example, the calculation of 
the 16 pixels can be halved to eight times, offering an 
advantage of being able to considerably reduce the calculation 
10 amount for the exposure control. 

EMBODIMENT 4 

Fig. 10 is a block diagram showing a configuration of 
an embodiment 4 of the image processing unit in accordance 

15 with the present invention. In Fig. 10, the same reference 
numerals designate the same components as those of the 
embodiment 1 as shown in Fig. 1, and the description thereof 
is omitted here. In Fig. 10, the reference numeral 9a 
designates a signal processing circuit having a gain 

20 correction function. 

Next, the operation of the present embodiment 4 will be 
described. 

The fixed length decoding circuit 7 reads the fixed length 
coded data stored in the coded image memory 6, carries out 

25 the normal fixed length decoding, and supplies the decoded 
image data to the signal processing circuit 9a. On the other 
hand, the system controller 18 reads from the coded image 
memory 6 the average levels LA, and supplies them to the signal 
processing circuit 9a. The signal processing circuit 9a 

30 carries out, in conjunction with the other signal processings, 
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the calculation for the gain correction on the basis of the 
average levels LA. 

According to the present embodiment 4, an advantage can 
be achieved of being able to control the gain correction 
5 coefficient in accordance with the data after the signal 
processings produced by the signal processing circuit 9a, in 
addition to the advantage of the foregoing embodiment 3. 

EMBODIMENT 5 

10 Fig. 11 is a block diagram showing a configuration of 

an embodiment 5 of the image processing unit in accordance 
with the present invention. In Fig. 11, the same reference 
numerals designate the same components as those of the 
embodiment 1 as shown in Fig. 1, and the description thereof 

15 is omitted here. In Fig. 11, the reference numeral 4 

designates a pixel rearrangement circuit that sorts the RGB 
color signals of pixels output from the A/D converter 3, and 
outputs a batch of unit blocks of respective color components 
for coding; 8 designates a reverse rearrangement circuit for 

20 carrying out reverse sorting of the pixel signals such that 
they are rearranged in the" same order as they are read out 
in the scanning direction, by sorting the data decoded by the 
fixed length decoding circuit 7 in the order reverse to the 
sorting order of the pixel rearrangement circuit 4; and 18a 

25 designates a system controller for carrying out level 

correction control of individual color signals based on the 
A/D converted image signals that undergo the fixed length 
coding . 

Next, the operation of the present embodiment 5 will be 
30 described. 
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The image pickup apparatus 1 captures the image of a 
subject, and outputs image signals corresponding to the 
incident light filtered by the color filters as shown in Fig. 
2, pixel by pixel in the scanning direction in a dot serial 
manner. The color image signals are amplified by the analog 
signal processing circuit 2, and filtered to remove their noise 
components. Subsequently, they are converted into digital 
signals through the A/D converter 3, and are supplied to the 
pixel rearrangement circuit 4 . 

The pixel rearrangement circuit 4, alternately using two 
line buffers not shown, sequentially writes into a first line 
buffer the image data of a scanning line, and at the same time 
reads from a second line buffer the image data with changing 
the read addresses, thereby carrying out the sorting of the 
15 image data. The image data, which is rearranged by the pixel 
rearrangement circuit 4 in such a manner that each color 
component forms a batch, is converted into fixed length coded 
data through the fixed length coding by the fixed length coding 
circuit 5. The fixed length coded data is stored in the coded 

20 image memory 6. 

Incidentally, the image signals output from the image 
pickup device, on which a plurality of color filters are 
disposed, exhibit different output level characteristics for 
the individual color signals because of the difference in the 
25 spectral sensitivity characteristics of the color filters. 
For example, when shooting an achromatic color subject with 
a pickup device with three RGB color filters, although the 
output levels of the color signals are expected to match with 
each other, they can differ in practice as shown in Fig. 12 (a) 
because of the difference in the spectral sensitivity 
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characteristics of the color filters. In view of this, the 
system controller 18a carries out a while balance processing 
to correct the levels' of the color signals so as to adjust 
them as shown in Fig. 12(b). The level correction of each 
color signal is based on the principle that the spectral 
characteristics of a common subject spreading over a wide range 
will become achromatic. Thus, the system controller 18a uses 
without change the image data as picked up in the level 
correction processing. In other words, if no sensitivity 
difference is present in the color filters when capturing 
* individual color image signals in a particular range, the 

Q levels of the color signals are expected to match with each 

2 other. Thus, the deviation in the level is corrected as the 

'f*. sensitivity difference. Accordingly, the white balance 

processing uses, as the AE processing, the integrals of the 
output signals of the image pickup device. 

In the present embodiment 5, the fixed length coding 
circuit 5 carries out fixed length coding of the data sorted 
by the pixel rearrangement circuit 4 such that the color signal 
components in each unit block become the same color, thereby 
achieving the fixed length coding for each color signal. As 
a result, the average level LA can be extracted from each color 
signal. For example, when G has the highest sensitivity in 
the three colors, the system controller 18a calculates a level 
25 correction coefficient Kr for the R signal by Kr = SG/SR, and 
a level correction coefficient Kb for the B signal by Kb = 
SG/SB, where SR, SG and SB are integrals of the average levels 
LA of the R, G and B signals, respectively. Then, the RGB 
signal levels can be adjusted by multiplying the R and B signals 
30 in the picked-up image signals by Kr and Kb, respectively. 
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The present embodiment 5 implements the real-time white 
balance processing by carrying out the correction processing 
during the sorting of the pixels for bringing the same color 
components together into a unit block. The calculation of 
the level correction coefficients Kr and Kb can be achieved 
by recursively correcting the already corrected signals with 
providing a delay, or can be completed at once by calculating 
them from the uncorrected image signals to terminate the white 
balance processing in a short time. Besides, it is not 
necessary to match the levels of the RGB signals . For example, 
they can be provided with level differences as needed such 
as correcting the color temperature of a light source. 

Although the reproduction method of the image signals 
of the present embodiment 5 is basically the same as that of 
15 the foregoing embodiments 1-3, since the present embodiment 
5 carries out the fixed length coding after the pixel sorting, 
the pixel reverse rearrangement circuit 8 performs the sorting 
opposite to that of the pixel rearrangement circuit 4 on the 
data decoded by the fixed length decoding circuit 7 so that 
20 the pixel signals are rearranged in the same order as the pixel 
signals read out in the scanning direction. The subsequent 
operation is the same as that of the embodiments 1-3. 

Thus, the present embodiment 5 offers an advantage of 
being able to reduce, by extracting average levels LA of 
25 individual signals, the amount of calculation for obtaining 
the level correction coefficients of the individual color 
signals . 
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EMBODIMENT 6 

Fig. 13 is a block diagram showing an embodiment 6 of 
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the image processing unit in accordance with the present 
invention. In Fig. 13 , the same reference numerals designate 
the same components as those of the embodiment 5 as shown in 
Fig. 11, and the description thereof is omitted here. In the 
5 present embodiment 6, the system controller 18a reads out the 
fixed length coded data stored in the coded image memory 6 
to carry out the level correction of each color signal. 

Next, the operation of the present embodiment 6 will be 
described. 

10 In the present embodiment 6, the data, which undergoes 

the pixel sorting by the pixel rearrangement circuit 4 and 
the fixed length coding by the fixed length coding circuit 
5, is temporarily stored in the coded image memory 6 as the 
fixed length coded data. The system controller 18a reads out 

15 the average levels LA of the individual color signals stored 
in the coded image memory 6, and carried out the white balance 
processing as in the foregoing embodiment 5. 

Thus, since the present embodiment 6 stores in the coded 
image memory 6 the coded data associated with the entire screen, 

2 0 it has an advantage of being able to implement a processing 
that can set the integration conditions flexibly by software 
or the like, for example. 

EMBODIMENT 7 

25 Fig. 14 is a block diagram showing an embodiment 7 of 

the image processing unit in accordance with the present 
invention. In Fig. 14, the same reference numerals designate 
the same components as those of the embodiment 6 as shown in 
Fig. 13, and the description thereof is omitted here. In Fig. 

30 14 , the reference numeral 7b designates a fixed length decoding 
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circuit that can achieve decoding in conjunction with the level 
correction of the individual color signals* 

Next, the operation of the present embodiment 7 will be 
described. 

5 In the present embodiment 7, the data, which is subjected 

to the sorting by the pixel rearrangement circuit 4 and to 
the fixed length coding by the fixed length coding circuit 
5, is temporarily stored in the coded image memory 6. The system 
controller 18a reads from the coded image memory 6 the average 

10 levels LA, and supplies them to the fixed length decoding 
circuit 7b with the level correction coefficients Kr and Kb, 
The fixed length decoding circuit 7b carries out the level 
correction of individual color signals on the basis of the 
average levels LA, and the level correction coefficients Kr 

15 and Kb, followed by the normal fixed length decoding. 

Thus, the present embodiment 7 has an advantage of being 
able to reduce the amount of the calculation for the level 
correction . 

20 EMBODIMENT 8 

Fig. 15 is a block diagram showing an embodiment 8 of 
the image processing unit in accordance with the present 
invention. In Fig. 15, the same reference numerals designate 
the same components as those of the embodiment 6 as shown in 

25 Fig. 13, and the description thereof is omitted here. In Fig. 
15, the reference numeral 9b designates a signal processor 
for carrying out the level correction of respective color 
signals after the normal signal processing. 

Next, the operation of the present embodiment 8 will be 

30 described. 
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In the present embodiment 8, the fixed length decoding 
circuit 7 reads from the coded image memory 6 the fixed 
length coded data, and carries out the fixed length 
decoding thereof, and the pixel reverse rearrangement 
5 circuit 8 rearranges the image data, which undergoes the 
fixed length decoding, in the same order as the image data 
read in the direction of the scanning line. Subsequently, 
the signal processing circuit 9b carries out the normal 
signal processing such as the pixel interpolation, the gray 

10 level correction and the gamma correction, followed by the 
level correction of the color signals, and supplies the 
image data to the frame memory 10. The subsequent 
operation is the same as that of the foregoing embodiments 
as described above. 

15 Thus, the present embodiment 8 has an advantage of 

being able to control the level correction coefficients 
by referring to the data after the signal processing. 

Incidentally, the system controller 18a can have a 
function to change the number of unit blocks to be processed, 

20 thereby limiting the amount of the calculation or the 
calculation duration . 

Furthermore, the system controller 18a can have a 
function to arbitrary set the location of the unit blocks to 
be processed, thereby neglecting from consideration the 

25 peripheries of the screen at which light amount can be 
insufficient due to lens characteristics. 

Moreover, the system controller 18a can have a function 
to arbitrary set the range of levels of the unit blocks to 
be processed in order to use only signals in a specified 

30 luminance range, thereby avoiding using low luminance level 
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signals susceptible to noise or high luminance level signals 
with different saturation characteristics. 
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What is claimed is: 

1- An image processing unit comprising: 

an A/D converter for carrying out A/D conversion of image 
signals output from an image pickup apparatus that picks up 
5 an image and converts it into electrical signals, and for 
outputting A/D converted image signals as image data; 

a fixed length coding circuit for dividing the image data 
into unit blocks , each consisting of a predetermined number 
of pixels, and for coding the pixels in each unit block after 
10 obtaining an average level of the pixels in the unit block; 
and 

an exposure controller for calculating a luminance level 
of a whole set of pixel data in the unit blocks by integrating 
the average levels of the unit blocks, and for controlling 
15 exposure of the image pickup apparatus such that the luminance 
level of the image data matches a predetermined level. 

2. The image processing unit according to claim 1, further 
comprising a coded image memory for storing fixed length coded 

20 data output from the fixed length coding circuit, the fixed 
length coded data including the average levels of the unit 
blocks . 

3. The image processing unit according to claim 2, further 
25 comprising a fixed length decoding circuit for reading from 

said coded image memory the fixed length coded data, for 
calculating a luminance level of the image data from the 
average levels, and for carrying out fixed length decoding 
of the fixed length coded data with performing gain correction 
30 for adjusting the luminance level of the image data to a 
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predetermined level . 

4. The image processing unit according to claim 2, further 
comprising : 

5 a fixed length decoding circuit for reading from said 

coded image memory the fixed length coded data, and for 
carrying out fixed length decoding of the fixed length coded 
data; and 

a signal processor for carrying out r using the average 
10 levels, gain correction of image data output from said fixed 
length decoding circuit. 

5. An image processing unit comprising: 

an A/D converter for carrying out A/D conversion of image 
15 signals output from an image pickup device that picks up an 
image and converts it into electrical signals including a 
plurality of color components, and for outputting A/D 
converted image signals as image data; 

a pixel rearrangement circuit for sorting the image data 
2 0 output from said A/D converter such that each color component 
is arranged in a unit block, each of which consists of a 
predetermined number of pixels; and 

a fixed length coding circuit for coding the pixels in 
each unit block after obtaining an average level of the pixels 
25 in the unit block. 

6. The image processing unit according to claim 5, further 
comprising an exposure controller for calculating a luminance 
level of a whole set of pixel data in the unit blocks by 

30 . integrating the average levels of the unit blocks, and for 



32 



controlling exposure of the image pickup apparatus such that 
the luminance level of the image data matches a predetermined 
level. 



5 7. The image processing unit according to claim 5, further 
comprising signal level correction means for correcting, 
using the average levels of the unit blocks, relative signal 
levels of the respective color components. 



10 8* The image processing unit according to claim 7, further 
comprising a coded image memory for storing fixed length coded 
data including the average level of each unit block, the fixed 
length coded data being output from the fixed length coding 
circuit . 

15 

9. The image processing unit according to claim 8, wherein 
said signal level correction means reads from said coded image 
memory the average levels, calculates from the average levels 
luminance levels of the pixel data in the unit blocks of the 

20 respective color components, and corrects signal levels of 
the color components such that the luminance levels of the 
color components match with each other. 

10. The image processing unit according to claim 8, further 
25 comprising a fixed length decoding circuit for reading from 

said coded image memory the fixed length coded data, and for 
carrying out fixed length decoding of the fixed length coded 
data with performing gain correction for adjusting, for each 
color component, the luminance level of the image data to a 
30 predetermined level by using the average levels- 
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11. The image processing unit according to claim 8, further 
comprising : 

a fixed length decoding circuit for reading from said 
5 coded image memory the fixed length coded data, and for 

carrying out fixed length decoding of the fixed length coded 
data; and 

a signal processor for carrying out, for each color 
component, signal level correction of image data output from 
10 said fixed length decoding circuit by using the average levels . 

12. The image processing unit according to claim 1, further 
comprising selecting means for selecting a number of the unit 
blocks - 

15 

13. The image processing unit according to claim 1, further 
comprising selecting means for selecting a location of the 
unit blocks . 

20 14. The image processing unit according to claim 1, further 
comprising selecting means for selecting the unit blocks in 
accordance with their average levels . 

15. The image processing unit according to claim 7, further 
25 comprising selecting means for selecting a number of the unit 
blocks . 
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16. The image processing unit according to claim 7, further 
comprising selecting means for selecting a location of the 
unit blocks . 
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17. The image processing unit according to claim 7 , further 
comprising selecting means for selecting the unit blocks in 
accordance with their average levels. 

5 

18. An image processing method of reproducing a picked-up 
still image, said method comprising the steps of: 

carrying out A/D conversion of image signals output from 
an image pickup apparatus, and for outputting A/D converted 

10 image signals as image data; 

dividing the image data into unit blocks, each consisting 
of a predetermined number of pixels, and coding the pixels 
in each unit block after obtaining an average level of the 
pixels in the unit block; and 

15 calculating a luminance level of a whole set of pixel 

data in the unit blocks by integrating the average levels of 
the unit blocks, and controlling exposure of the image pickup 
apparatus such that the luminance level of the image data 
matches a predetermined level. 

20 
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Abstract of the Disclosure 

An image processing unit is provided which can solve a 
problem of a conventional image processing unit in that the 
control for coding image data is complicated and its circuit 
5 scale is large. The image pickup apparatus picks up an image 
of a subject and converts it to electric signals, thereby 
outputting image signals- An A/D converter A/D converts the 
image signals, and outputs them as image data. A fixed length 
coding circuit breaks down the image data into small unit 

10 blocks consisting of a predetermined number of pixels, and 
carries out fixed length coding after obtaining the average 
level of the pixel data in each unit block. An exposure 
controller calculates a luminance level of a whole set of pixel 
data in the unit blocks, and controls the exposure of the image 

15 pickup apparatus so that image data becomes a predetermined 
luminance level . 
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PTO/S8/106(S^6) 
Approved for um through 9/30/98. OMB 0651-0032 
Pattnt and Tridemark Office; U.S. DEPARTMENT OF COMMERCE 
Under the Paperwork Reduction Act of 1995, no person* ire required to retpond to * collection of Information unless K dttptayi a valid OMB control number. 



Declaration and Power of Attorney For Patent Application 
Japanese Language Declaration 

T&<D&%i <OS£$!# £ It, &&£lTc0iI 9 Sff L 4 i". As a below named inventor, I hereby declare that: 

f&D&Bf* 3&W3f, ©Jtr^TI2©^<Ofic40^^IE3fe$tt/ My residence, post office address and citizenship are as stated 
fca 9 next to my name. 

TTE^^I^©35WKlBIIL'C0[^KHlC|5!ft$i^, WfFffJIS * believe i am the original, first and sole inventor {if only one name 
LT^5389!r*f8Flcov*-C, Si 5 *H!j3a»oiS— ©ftW* (T is listed below) or an original, first and Joint inventor (if plural 
15 <D 2 s — o t> U < fiftSJ 5 names are listed below) of the subject matter which is claimed and 

k (TfS<O£#J$S}£g:cDS£0 ftCTV^i** for which a patent is sought on the invention entitled 

" IM&GE PROCESSING UNIT AND IMAGE PROCESSING 

METHOD WITH SIMPLIFIED EXPOSURE, GAIN 

AND WHITE BALANCE CONTROL" 

±&&Wi<D%iji8&- (TIEcOffi'Tix^^Ov^-CV^V^^i. the specification of which is attached hereto unless the following 
2fcSFfcK#) f£* box is checked: 



□ *HfflM#**fcr±1$#»;g&# Q was Hied on 

!HI3gfci3H#"^ £ k L * as United States Application Number or 

(ffiS "J" fclfTjE^il^Lfc* PCT international Application Number 

and was amended on 

{if applicable). 



1 hereby state that I have reviewed and understand the contents of 
the above identified specification, including the claims, as 
amended by any amendment referred to above. 



flirt* 3£f?^I'j&*fg 3 7 1^5 6 3g^^S£ £ k I acknowledge the duty to disclose information which is material to 
33 n , ^l?R^^0 ; 6■^t^l•o^/^-Cg^^f^^^^:^^i-^^^^ patentability as defined in Title 37, Code of Federal Regulations, 
Z k %tg£>-£~t. Section 1.56. 



Page 1 of 4 

Burden Hour Statement: This form is estimated to talc© 0.4 hours to complete. Time will vary depending upon the needs of the individual c&se. Any comments on the 
amount of time you are required to complete this form should be sent to the Chief Information Officer, Patent and Trademark Office, Washington, DC 2023 1 . DO NOT 
SEND FEES OR COMPLETED FORMS TO THIS ADDRESS. SEND TO: Commissioner of Patent* and Trademarks, Washington, DC 20231. 
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Approved for use through 9/30/98. OMB 0651-0032 
Patent and Trademark Office; U.S. DEPARTMENT OF COMMERCE 
Under the Paperwork Reduction Act of 1 995, no persons are required to respond to a coSectfon of information unless It displays a vaid OMB control number. 



Japanese Language Declaration 

(a*iitf) 



(b) 3te:36£T1E<7X *BM©Bo*ft< 4t>— *H*rffl 
3eUTV^aiMF***» 3 6 5 (a) gffc*-f < SSSfflS, X 



Prior Foreign Application^} 

10-176418 

(Number) 
<**) 

(Number) 
(#*> 



Japan 



(Country) 



(Country) 
CM*) 



(Application No.) 
(fflfi##) 



(Filing Date) 
(fflKB) 



fttt. TE<E)*H«A1S 3 5S12 0*ltI^tTlB©* 

7t, ^aajHcD^jtt^asia^rts^s^iBs sail i 2* 



(Application No.) 



(Application No. 



(Filing Date) 

(Filing Date) 
(fcHKB) 



18l»10 0 1feSf^, fi&*fcr±}*J&* t>U<tt^ 
<&#»tt* s *;bfr*;i4£S«U* ioT£CfcJbfa<Z>r4 



( hereby claim foreign priority under Title 35, United States Code, 
Section 113 (a)-(d) or 365(b) of any foreign application^) for patent 
or inventor's certificate, or 365(a) of any PCT Internationa/ 
application which designated at least one country other than the 
United States, listed below and have also identified below, by 
checking the box, any foreign application for patent or inventor's 
certificate, or PCT international application having a filing date 
before that of the application on which priority is claimed. 

Priority Not Claimed 
«#r**H*L 

23/June/1998 

(Day/Month/Year Filed) 



(Day/Month/Year Filed) 

I hereby claim the benefit under Title 36, United States Code, 
Section 119(e) of any United States provisional application {s} listed 
below. 



(Application No.) 



(Fiiing Date) 



I hereby claim the benefit under Title 35, United States Code, 
Section 120 of any United States appJicatlon(s), or 385(c) of any 
PCT International application designating the- United States, listed 
below and, insofar as the subject matter of each of the claims of 
this application is not disclosed in the prior United States or PCT 
International application in the manner provided by the first 
paragraph of Title 35, United States Code Section 112, I 
acknowledge the duty to disclose information which is material to 
patentability as defined in Title 37, Code of Federal Regulations, 
Section 1.56 which became available between the filing date of the 
prior application and the national or PCT International filing date of 
application. 

(Status: Patented, Pending, Abandoned) 

(Status: Patented, Pending, Abandoned) 

t hereby declare that ail statements made herein of my own 
knowledge are true and that all statements made on information 
and belief are believed to be true; and further that these 
statements were made with the knowledge that willful false 
statements and the like so made ara punishable by fine or 
imprisonment, or both, under Section 1001 of TitJe 18 of the 
United States Code and that such willful false statements may 
jeopardize the validity of the application or any patent issued 
thereon. 
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PTO/S 8/1 06 (6-96) 
Approved for use through 9/30/96. OMB 0651-OC32 
Pitant ind Trademark Office; U.S. DEPARTMENT OF COMMERCE 
Under the Paperwork Reduction Act of 1995, no por&ons arc required to respond to a ccfloction of Information untew ft display* a valid OMB control number, 

Japanese Language Declaration 



TERRELL C. BIRCH {Reg. No. 19.3325 JAMES M. S LATTERY {Reg. Ma. 28.380) 

RAYMOND C. STEWART {Reg. No. 21.0661 BERNARD. L SWEENEY (Reg. No. 24,448) 

JOSEPH A. KOLASCH (Reg. No. 22.4631 MICHAEL K. MUTTER {Reg. No. 23.630) 

ANTHONY L BIRCH [Reg. No. 26,1221 CHARLES G0RENSTEIN (Reg. No. 29.2711 



POWER OF ATTORNEY: As a named Inventor, I hereby appoint 
the following attomey(e) and/or agent(s) to prosecute this 
application and transact all business in the Patent and Trademark 
Office connected therewfth (list name and regis (radon number) 

GERALD M. MURPHY (Reg. Mo. 28,977] MARC S. WEINER {Reg. Mo. 32,181) 

LE0NAR0 R. SVENSSON {Reg. Mo. 30,330) ANDREW F. REISH {Reg. No. 33.443) 

TERRY L CLARK {R«g. No. 32,544) JOE M. MUNCY (Reg. No. 32,334) 

ANOREW 0. MEiXLE (Reg. No. 32,3881 C. JOSEPH FARACI (Reg. No. 32.350) 



SfSSszs^^fe Send Correspondence to: 



8iRCH. STEWART, K0LASCH & BIRCH. LLP 
P.O. 30X 747 

FALLS CHURCH, VA 22040-0747 
TEL: (703) 205-3000 



G£SSS3S3[S5fe : (& Direct Telephone Caiis to: {name and telephone number) 



8IRCH, STEWART, KOLASCH & BIRCH, LLP 
TEL: (703) 205-8000 



Hi— *fct±#— &m#& 


Full name of sole or first inventor 

Narihiro MATOBA. 




Inventor's signature * Date 

^vJlvk* , ^ wfPW June 11, 1999 


mm 


Residence 

Tokyo, Japan 


mm 


Citizenship 

Japanese 




Post Office Address 
c/o MITSUBISHI DENKI KABUSHIKI KAISHA, 


2-3 , Marunouchi 2-chorne, Chiyoda-ku, 
Tokyo 100-8310 Japan 




Full name of second joint inventor, if any 
Kazuto TERADA 




Second inventor's signature Date 

Ka^uTo 11, 1999 


mm 


Residence 

Tokyo, Japan 




Citizenship 

Japanese 




Post Office Address 

c/o Microsoft GomoraHon. 


2-2-1, Yoyogi, Shibuya-ku, Tokyo 
151-8583 Japan 



(J£- WlS$cD^!133Sf!#tCOV^T tsfSJ^clC 1542 b, 5£2r"t" (Supply similar information and signature for third and subsequent 
i) joint inventors.) 
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Full name of third joint inventor, if any, 
Masashi T.AMQRA 








Third Inventor ! s signature Date 
^O&Az/tZ Tatnui^ June 11 ' 1999 


m 






Residence 

Tokyo, Japan 


mm 






Citizenship 
Japanese 








Post Office Address 

c/o MITSUBISHI DENKI KABUSHIKI KAISHA, 


2-3, Marunouchi 2-chome, Chiyoda-ku, 
Tokyo -100-83 10 Japan 








Full name of fourth joint inventor, if any, 








Fourth Inventor's signature Date 








Residence 


5 |jf§ Citizenship 








Post Office Address 










Full name of fifth joint inventor, if any, 








Fifth Inventor's signature Date 








Residence 


|5li§ Citizenship 








Post Office Address 
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